Skip to content

Conversation

stefnestor
Copy link
Contributor

👋 howdy, team! Links node does not match index setting Allocation message to troubleshooting doc as part of linking common Elasticsearch errors(/troubleshooting messages) to their docs project.

Similar ballpark previous: #110911, #110993, #111315, #110449, #113561, #113656, #113657

@stefnestor stefnestor added >enhancement >non-issue :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. auto-backport Automatically create backport pull requests when merged Supportability Improve our (devs, SREs, support eng, users) ability to troubleshoot/self-service product better. v8.17.0 labels Oct 23, 2024
Copy link
Contributor

Documentation preview:

@elasticsearchmachine elasticsearchmachine added v9.0.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Oct 23, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@stefnestor
Copy link
Contributor Author

🙋‍♀️ May I have feedback on why CI's failing? Local tests are either successful or an intermittent cluster timeout unrelated to my changes as checked from pre-PR-checklist:

./gradlew spotlessApply && ./gradlew internalClusterTest && ./gradlew precommit
./gradlew ":server:internalClusterTest" --tests "org.elasticsearch.action.admin.cluster.allocation.ClusterAllocationExplainIT"

Vs this pipeline is failing

Tests with failures:
 - org.elasticsearch.smoketest.DocsClientYamlTestSuiteIT.test {yaml=reference/cluster/allocation-explain/line_91}
...
» [2024-10-24T17:41:11,328][WARN ][r.suppressed             ] [node-0] path: /_search, params: {error_trace=true}, status: 503 Failed to execute phase [query], all shards failed; shardFailures {[_na_][idx][2]: org.elasticsearch.action.NoShardAvailableActionException
»  }{[_na_][idy][42]: org.elasticsearch.action.NoShardAvailableActionException
»  }{[_na_][idy][43]: org.elasticsearch.action.NoShardAvailableActionException
»  }

But I don't know how to fix ES shard issues within I assume a pipeline test cluster other than git merge master 🙏

@DaveCTurner
Copy link
Contributor

The failure is here:

DocsClientYamlTestSuiteIT > test {yaml=reference/cluster/allocation-explain/line_91} FAILED
    java.lang.AssertionError: Failure at [reference/cluster/allocation-explain:100]: Expected a map containing
         allocate_explanation: "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there."
              unassigned_info: a map containing
                         reason: "INDEX_CREATED"
                             at: "2024-10-24T18:04:49.325Z"
         last_allocation_status: "no"
                current_state: "unassigned"
    node_allocation_decisions: a list containing
                              0: a map containing
                   weight_ranking: <1>
                  node_attributes: a map containing
                    xpack.installed: "true"
            ml.allocated_processors: "32"
                    ml.max_jvm_size: "536870912"
                  ml.config_version: "12.0.0"
                  ml.machine_memory: "101237022720"
     ml.allocated_processors_double: "32.0"
                           testattr: "test"
           transform.config_version: "10.0.0"
                transport_address: "127.0.0.1:34703"
                            roles: a list containing
                                  0: "data"
                                  1: "data_cold"
                                  2: "data_content"
                                  3: "data_frozen"
                                  4: "data_hot"
                                  5: "data_warm"
                                  6: "ingest"
                                  7: "master"
                                  8: "ml"
                                  9: "remote_cluster_client"
                                 10: "transform"
                        node_name: "node-0"
                         deciders: a list containing
                                  0: a map containing
                             decision: "NO"
                              decider: "filter"
                          explanation: expected "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]" but was "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]; for more information, see [https://www.elastic.co/guide/en/elasticsearch/reference/master/cluster-allocation-explain.html#allocation-explain-setting-conflict]"
                    node_decision: "no"
                          node_id: "rGfCdhbUQ0aQbGEwOb_93w"
                        index: "my-index-000001"
                        shard: <0>
                 can_allocate: "no"
                      primary: <true>

note the mismatch at explanation: expected "..." but was "..." because this line in the docs needs fixing:

"explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]" <6>

@elasticsearchmachine elasticsearchmachine added v8.18.0 Team:Distributed Coordination Meta label for Distributed Coordination team and removed v8.17.0 labels Nov 20, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-obsolete (Team:Distributed (Obsolete))

@stefnestor
Copy link
Contributor Author

Um, I'm not figuring this out, am behind on other things, & it's the lowest priority one on my list, so I'm just going to close & can try again another day. Thanks for your help & sorry for the noise! 💚

@stefnestor stefnestor closed this Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >enhancement external-contributor Pull request authored by a developer outside the Elasticsearch team >non-issue Supportability Improve our (devs, SREs, support eng, users) ability to troubleshoot/self-service product better. Team:Distributed Coordination Meta label for Distributed Coordination team Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants